Papers with gating mechanism
Associative Multichannel Autoencoder for Multimodal Word Representation (D18-1)
Copied to clipboard
| Challenge: | Existing models that represent word meanings from word co-occurrences ignore associations between modalities and lack ability to transfer information between . |
| Approach: | They propose a novel associative multichannel autoencoder that integrates textual, visual and auditory inputs to learn multimodal word representations. |
| Outcome: | The proposed model outperforms strong unimodal models and state-of-the-art models on six benchmark concepts similarity tests. |
Modeling Coherence for Neural Machine Translation with Dynamic and Topic Caches (C18-1)
Copied to clipboard
| Challenge: | Current neural machine translation systems translate a text sentence-by-sentence, ignoring cross-sentent links and dependencies. |
| Approach: | They propose a cache-based approach to modeling coherence for neural machine translation . they capture contextual information either from recently translated sentences or the entire document . |
| Outcome: | The proposed model improves on state-of-the-art translation models on many languages . it captures contextual information from recently translated sentences or the entire document . |
A Mixture-of-Experts Model for Antonym-Synonym Discrimination (2021.acl-short)
Copied to clipboard
| Challenge: | Anatomy-synonymy discrimination (ASD) is a crucial problem in lexical semantics and is difficult to distinguish between antonyms and synonyms. |
| Approach: | They propose a divide-and-conquer strategy where localized experts focus on their own domains to learn their specialties. |
| Outcome: | The proposed method achieves state-of-the-art performance on the Antonymy-synonymy discrimination task. |
Knowledge-Guided Paraphrase Identification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for paraphrase identification (PI) are limited due to lack of professional knowledge. |
| Approach: | They propose to leverage Wikipedia knowledge to accurately identify paraphrases by mining outline knowledge of given sentences from Wikipedia. |
| Outcome: | The proposed framework outperforms state-of-the-art models on two public datasets: PARADE and clinicalSTS2019. |
Long Short-Term Memory as a Dynamically Computed Element-wise Weighted Sum (P18-2)
Copied to clipboard
| Challenge: | LSTMs were introduced to combat vanishing gradients in simple RNNs by augmenting them with gated additive recurrent connections. |
| Approach: | They propose to decouple the LSTM’s gates from the embedded RNN and create a new class of RNNs where the recurrence computes an element-wise weighted sum of context-independent functions of the input. |
| Outcome: | The proposed model performs as well as an LSTM on a range of problems, strongly suggesting that the gates are doing much more in practice than just alleviating vanishing gradients. |
Incorporating Syntax and Semantics in Coreference Resolution with Heterogeneous Graph Attention Network (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing neural coreference resolution models lack syntactic and semantic information . however, such information has been shown to benefit other tasks. |
| Approach: | They propose a graph-based model that incorporates syntactic and semantic structures of sentences. |
| Outcome: | The proposed model incorporates syntactic and semantic structures of sentences. |
ReGLA: Refining Gated Linear Attention (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) are known for their computational and storage requirements due to the quadratic computation complexity of softmax attention. |
| Approach: | They propose to reduce the quadratic computation complexity of softmax attention by using feature maps, normalization and the gating mechanism to improve performance. |
| Outcome: | The proposed model outperforms existing gated linear attention models in extensive tasks including training from scratch and post-linearization with continual pre-training. |
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia (P18-1)
Copied to clipboard
| Challenge: | Existing models that only take into account sentence-level information do not generate question-answer pairs. |
| Approach: | They propose a neural network approach that incorporates coreference knowledge via a novel gating mechanism for paragraphlevel question generation. |
| Outcome: | The proposed model outperforms existing models on a Wikipedia article question-answer generation task. |
Entropy-Gated Branching for Efficient Test-Time Reasoning (2026.eacl-long)
Copied to clipboard
| Challenge: | Empirical results show that branching at low uncertainty points can improve reasoning capabilities of large language models . however, these methods require substantially more computational resources, causing errors in high-stakes domains . |
| Approach: | They propose an inference technique that selectively expands prediction sequences at points of high uncertainty. |
| Outcome: | Empirical results show that the proposed method improves accuracy by 22.6% over standard inference while operating 31%-75% faster across math benchmarks. |
A Tale of Two Linkings: Dynamically Gating between Schema Linking and Structural Linking for Text-to-SQL Parsing (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for text-to-SQL semantic parsing require strict structured prediction due to its application scenario where the output SQL will be sent to an executor program directly. |
| Approach: | They propose to use schema linking and structural linking to link NL to the database schema. |
| Outcome: | The proposed method shows significant gains on the Spider dataset. |
Bridging the Gap: Attending to Discontinuity in Identification of Multiword Expressions (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to identify discontinuous multiword expressions are limited in dealing with discontinuous occurrences. |
| Approach: | They propose a method to tag Multiword Expressions using a language-independent deep learning architecture to target discontinuity. |
| Outcome: | The proposed model outperforms baseline models on a multilingual dataset and scores higher than baseline models. |
Gated Transformer for Robust De-noised Sequence-to-Sequence Modelling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Noisy texts are common in user-generated texts that appear abundant in social media platforms like SMS, online chat, email, blogs, wikis etc. |
| Approach: | They propose a sequence-to-sequence architecture that uses a gating mechanism to detect types of corrections required from English texts. |
| Outcome: | The proposed architecture performs better than non-gated models on machine translation and Summarization tasks. |
Removing Word-Level Spurious Alignment between Images and Pseudo-Captions in Unsupervised Image Captioning (2021.eacl-main)
Copied to clipboard
| Challenge: | Unsupervised image captioning is a challenging task that requires manual annotation. |
| Approach: | They propose a simple gating mechanism that is trained to align image features with the most reliable words in pseudo-captions. |
| Outcome: | The proposed method outperforms the previous methods without complex learning objectives. |
Encoding Gated Translation Memory into Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (MT) technology has made significant progress in the past few years. |
| Approach: | They propose a method to combine the strengths of TM and neural machine translation (NMT) they use a gating mechanism to balance the impact of the TM match on the NMT decoder . |
| Outcome: | The proposed method improves translation quality by over 10 BLEU points when fuzzy matches are higher than 50% on the UN corpus. |
Operation-guided Neural Networks for High Fidelity Data-To-Text Generation (D18-1)
Copied to clipboard
| Challenge: | Recent neural models for data-to-text generation generate descriptions that are not consistent with structured data. |
| Approach: | They propose a framework for data-to-text generation that uses symbolic operations to generate texts from structured data. |
| Outcome: | The proposed framework improves the fidelity of the generated texts to the input structured data. |
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient language model tuning (PELT) match the performance of fine-tuning with fewer trainable parameters. |
| Approach: | They propose a framework which integrates different PELT methods as submodules and learns to activate the ones that best suit the current data or task setup via gating mechanism. |
| Outcome: | The proposed framework outperforms fine-tuning methods on the GLUE benchmark and achieves 14% gains over the best individual PELT method. |
Incorporating Background Knowledge into Video Description Generation (D18-1)
Copied to clipboard
| Challenge: | Existing methods for video captioning focus on generating generic descriptions that lack contextual knowledge. |
| Approach: | They propose a method that uses video meta-data to retrieve topically related news documents for a video and extracts the events and named entities from these documents. |
| Outcome: | The proposed model is based on a news video dataset and is evaluated on it. |
Event Detection: Gate Diversity and Syntactic Importance Scores for Graph Convolution Neural Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on event detection (ED) have shown that the syntactic dependency graph can be employed in graph convolutional neural networks (GCNs) but the computation of the hidden vectors in such graph-based models is agnostic to the trigger candidate words, leaving irrelevant information for the trigger candidates. |
| Approach: | They propose a mechanism to filter noisy information in the hidden vectors of graph-based models based on the information from the trigger candidate. |
| Outcome: | The proposed model achieves state-of-the-art on two ED datasets. |
Decompose, Fuse and Generate: A Formation-Informed Method for Chinese Definition Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing definition generation methods take the source word as an indecomposable semantic unit, but in parataxis languages like Chinese, word meanings can be composed using the word formation process. |
| Approach: | They propose to use word formation features to enhance Definition Generation (DG) in Chinese to generate an explanatory text. |
| Outcome: | The proposed model enhances Definition Generation (DG) in Chinese by decomposing the word meaning into different semantic components. |
Task-Oriented Conversation Generation Using Heterogeneous Memory Networks (D19-1)
Copied to clipboard
| Challenge: | Existing memory networks do not perform well when leveraging heterogeneous information from different sources. |
| Approach: | They propose to use user utterances, dialogue history and background knowledge tuples to integrate external knowledge into a neural dialogue model. |
| Outcome: | The proposed model outperforms the state-of-the-art data-driven task-oriented dialogue models on real-world datasets. |
Towards Context-Robust LLMs: A Gated Representation Fine-tuning Approach (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) enhanced with external contexts face challenges in handling imperfect evidence. |
| Approach: | They propose a framework that can balance internal knowledge with external contexts . they propose gating mechanisms and low-rank representation adapters to adjust hidden representations based on a lightweight intervention function . |
| Outcome: | The proposed model can effectively balance internal knowledge with external context, similar to human cognitive processes. |
The Learnability of Model-Theoretic Interpretation Functions in Artificial Neural Networks (2026.findings-acl)
Copied to clipboard
| Challenge: | Entity vectors improve scores on basic event, while gated architectures benefit most. |
| Approach: | They extend entity-level semantic representations, modern architectures, principled competing event generation, extended systematicity tests and a two-dimensional difficulty analysis disaggregating results by modifier complexity. |
| Outcome: | The proposed model-theoretic interpretation functions generalize systematically to out-of-training-sample sentences. |
The Bidirectional Process Reward Model (2026.acl-long)
Copied to clipboard
| Challenge: | Process reward models (PRMs) assign fine-grained scores to intermediate reasoning steps within a solution trajectory. |
| Approach: | They propose a bidirectional evaluation paradigm that integrates a parallel evaluation stream alongside the L2R evaluation scheme and a gating mechanism to fuse the reward scores. |
| Outcome: | The proposed model surpasses unidirectional baselines in multiple benchmarks, LLM objectives and sampling policies. |
Leveraging Entity Information for Cross-Modality Correlation Learning: The Entity-Guided Multimodal Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Summarization with Multimodal Output (MSMO) is a new approach to produce a multimodal summary that integrates both text and relevant images. |
| Approach: | They propose an Entity-Guided Multimodal Summarization model that integrates both text and relevant images to produce a multimodal summary. |
| Outcome: | The proposed model integrates text-image and entity-image information and refines image selection through knowledge distillation from a pre-trained vision-language model. |
Highway Transformer: Self-Gating Enhanced Self-Attentive Networks (2020.acl-main)
Copied to clipboard
| Challenge: | Self-attention mechanisms have made striking state-of-the-art (SOTA) progress in various sequence learning tasks, attending to all the global contexts at different locations. |
| Approach: | They propose a gated component self-dependency units (SDU) that incorporates LSTM-styled gating units to replenish internal semantic importance within the multi-dimensional latent space of individual representations. |
| Outcome: | The proposed system could boost the performance of the Transformer modules by allowing them to skip connections and step towards suboptimal points during the optimization process. |
Hard Gate Knowledge Distillation - Leverage Calibration for Robust and Reliable Language Model (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation schemes focus on a teacher as a source of knowledge and a gauge to detect miscalibration of a student. |
| Approach: | They propose a method that uses a teacher model as a source of knowledge and a model as an error detector to detect miscalibration of a student. |
| Outcome: | The proposed scheme improves model generalization and significantly lowers calibration error. |
Extractive Medical Entity Disambiguation with Memory Mechanism and Memorized Entity Information (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on local optimal while ignoring sole-mention disambiguation boosted by richer context from other mentions’ disambiguating processes. |
| Approach: | They propose an approach to extracting medical entity disambiguation using memory mechanism and memorized entity information (M3E) they use a memory mechanism module that performs memory caching, retrieval, fusion and cross-network residual to aid the disambiguations of remaining mentions. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two benchmark datasets. |
DueT: Image-Text Contrastive Transfer Learning with Dual-adapter Tuning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Comparative learning models for vision and language models are gaining popularity . dueT trains only adapters inserted into pre-trained image and text encoders . |
| Approach: | They propose a transfer learning method for vision and language models built by contrastive learning that trains only adapters inserted into the frozen image and text encoders. |
| Outcome: | The proposed method outperforms fine-tuning, and the LoRA-based adapter method in English and Japanese domains. |
SMoP: Towards Efficient and Effective Prompt Tuning with Sparse Mixture-of-Prompts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prompt tuning has emerged as a successful parameter-efficient alternative to the full fine-tuning of language models. |
| Approach: | They propose a prompt tuning method that utilizes short soft prompts for efficient training and inference while maintaining performance gains typically induced by longer soft prompt. |
| Outcome: | The proposed method outperforms baseline methods while preserving memory usage. |
Emphasising Structured Information: Integrating Abstract Meaning Representation into LLMs for Enhanced Open-Domain Dialogue Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation metrics struggle to evaluate adversarial negative examples . existing metrics struggle in handling adversarials, resulting in low correlations with human judgments. |
| Approach: | They propose a framework that integrates AMR and domain-specific language models for automatic open-domain dialogue evaluation. |
| Outcome: | The proposed evaluation framework achieves strong correlations with human judgments across multiple datasets. |
Semantics-Aware Dual Graph Convolutional Networks for Argument Pair Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Argument pair extraction (APE) aims to extract interactive argument pairs from two separate passages. |
| Approach: | They propose to tackle the lexical and semantic relevance of arguments with a pre-trained Rouge-guided Transformer (ROT) by using a word graph and a gating mechanism to fuse two graphs. |
| Outcome: | The proposed approach achieves state-of-the-art on F1 score and significantly improves on existing best alternative. |
V-RoLoRA: RLVR-Driven MoE Routing for Steerable Pluralistic Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Current methods for steering large language models rely on prompt engineering or reasoning-time guidance. |
| Approach: | They propose a value-controllable pluralistic alignment framework enhanced with conditioned gating that dynamically directs the flow among multiple experts based on an input value or moral vector. |
| Outcome: | The proposed method outperforms prompt-based steering and multi-task PEFT benchmarks on two 8-billion-parameter backbones. |
GLA: Grounding Large Language Models in Molecular Hierarchy for Chemical Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing molecule-language models obscure the hierarchical organization of chemical semantics . Existing models rely on linear or uniform encodings, causing structural distortion . |
| Approach: | They propose a framework that integrates intrinsic molecular topology into large language models. |
| Outcome: | The proposed framework improves on cross-modal retrieval, captioning, and property prediction benchmarks. |
CIS-BWE: Chaos-Informed Speech Bandwidth Extension (2026.acl-long)
Copied to clipboard
| Challenge: | CIS-BWE introduces two chaos-informed discriminators for capturing the deterministic chaos from speech. |
| Approach: | They propose a novel adversarial Bandwidth Extension framework that introduces two chaos-informed discriminators for capturing the deterministic chaos from speech. |
| Outcome: | The proposed framework achieves better performance across nine subjective and objective evaluation metrics with a 40x reduction in discriminator size and overall 0.5x fewer parameters, establishing a new baseline in the BWE task. |